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DETAILED ACTION 



Drawings 



1. The drawings were received on July 11, 2003. These drawings are not acceptable. It is 
believed that applicant intended to add an arrow (line) pointing from the numeral 712 to the 
ellipse fitted to the candidate patch 710, rather than deleting the numeral (see page 11, line 25- 
27). The replacement Figures 3, 4, and 5 have been approved. 



2. Applicant's amendment to the specification and petition, filed July 15, 2003, with respect 
to the color drawings have been fully considered and are persuasive. The objection of the 
drawings has been withdrawn. 

3. Applicant's arguments, see pages 10-13, filed July 28, 2003, with respect to the 
rejection(s)of claim(s) 1, 4, 5, 8, 12, 15, 16, 19, and 23 under 35 USC 103(a) have been fully 
considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon 
further consideration, a new ground(s) of rejection is made in view of Cosatto et al. (USPN 
5,864,630, cited on appUcant's IDS). The examiner would Uke to point out that Kinjo is still 
relied upon for a portion of the claimed subject matter. On page 11, last paragraph, appUcant 
states "the Applicant respectfully disagrees with the interpretation that the 'changing differential 
values' analysis (see column 18-19 lines 65-17) equates to 'determining a first area wherein a 
color gradient has a low value' as claimed in claim 1." Merriam-Webster's Collegiate 
Dictionary (10^^ Edition) defines the word gradient as "1 a: the rate of regular or graded ascent or 
descent: INCLINATION... 2: change in the value of a quantity (as temperature, pressure, or 
concentration) with change in a given variable and esp. per unit distance in a specified direction, 



Response to Arguments 
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3: the vector sum of the partial derivatives with respect to the three coordinate variables x, and 
z of a scalar quantity whose value varies from point to point. . ." Although Kinjo does not 
explicitly use the word "gradient," the examiner believes this is exactly what is being done when 
-/"density values of the respective pixels in the regions are differentiated along a plurality of 
mutually different directions. . . a determination is made as to whether or not there is a region in 
which differential values in at least one of the plurality of directions changes regularly [high 
gradient value] . . . YES is given as the answer in the case of the region whose density changes 
with a predetermined pattern or the region whose density shows a repetition of a predetermined 
pattern of change. Since it can be determined that such a region has a low probability of being a 
region corresponding to a human figure, . .the region in which differential values change 
regularly is excluded in Step 162 from the subject of determination of the region representing the 
contour of the head of a human figure" (emphasis added; column 18, line 65-column 19, line 17). 
The examiner believes that the latter step of excluding the region where differential values 
change regularly is equivalent to "determining a first area wherein a color gradient has a low 
value" as stated in claim 1 . The examiner would also like to point out that Kinjo does not limit 
the search of a shape pattern representing the internal structure of the face to a pair of eyes, but 
may also include using a nose, mouth, pair of eyebrows, or the like or combination thereof 
(column 39, line 45-51). 

Claim Rejections - 35 USC §112 
4. The following is a quotation of the second paragraph of 35 U.S.C. 1 12: 

The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the 
subject matter which the applicant regards as his invention. 
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5. Claim 34 is rejected under 35 U.S.C. 1 12, second paragraph, as being indefinite for 
failing to particularly point out and distinctly claim the subject matter which appHcant regards as 
the invention. Claim 34 recites the limitation "said at least one candidate patch" in line 8-9. 
There is insufficient antecedent basis for this limitation in the claim. 

Claim Rejections - 35 USC § 103 

6. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

7. Claims 1-5, 8, 12-16, 19, 23-27, 30, 34-38, 41, and 45 are rejected under 35 U.S.C. 
103(a) as being unpatentable over Kinjo (USPN 5,629,752, cited on applicant's IDS) in view of 
Cosatto et al. (USPN 5,864,630, cited on appUcant's IDS), hereafter Cosatto. 

8. Regarding claims 1, 23, and 45, Kinjo discloses an apparatus and method of determining 
at least one candidate patch for human faces in a color graphic image (column 19, line 18-25), 
comprising: determining a first area wherein a color gradient (changing differential values) has a 
low value (exclude the regions in which differential values change regularly; column 18, line 65 
to column 19, hne 17); determining a second area (white region) wherein an intensity value has a 
high value (corresponding to a low pixel density value; column 17, line 33-42). 

9. Kinjo does not disclose performing a logical AND on said first area and said and second 
areas to create a third area, or selecting portions of the third area with suitable hue saturation to 
form said at least one candidate patch. Cosatto discloses a multi-modal method for locating 
objects in images using information obtained from several channels to evaluate the quality of 
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individual features, as well as combinations of features including shape, color, and motion 
(column 5, line 47-54). It is also disclosed that the data produced by each channel comprises a 
list of areas which may contain head outlines and facial features (column 5, line 66-column 6, 
line 2), which are compared, evaluated, and integrated by a classifier which assigns a measure of 
confidence for each feature and each combination of features to determine which channel or 
combination should be used to arrive at the final result (column 6, line 20-34). Cosatto does not 
explicitly disclose that a logical AND is performed on a first area and a second area to create a 
third area, it is disclosed that areas from each feature channel are compared, evaluated, and 
integrated such that areas with the desired features are found having the combined features 
(column 6, line 58-66). Cosatto further discloses that an area with suitable hue saturation (skin 
color) is selected to form at least one candidate patch (column 13, line 14-29). It would have 
been obvious to one of ordinary skill in the art at the time the invention was made to locate 
objects in images using areas found using combined results ("logical AND") from several feature 
channels and selecting areas with suitable hue saturation to form at least one candidate patch as 
taught by Cosatto in order to enable a tracking system to extract enhanced detail from the 
collected images thereby improving the accuracy and robustness of the tracked output over that 
of simple color segmentation as well as providing information to the tracking system which may 
be used to train the system for subsequent tracking (column 2, line 8-28). 
10. Regarding claim 12, Kinjo discloses a system configured to determine at least one 
location of a human face in a color graphic image (column 19, line 18-25), comprising: a color 
gradient map configured to indicate true where a color gradient (changing differential values) has 
a low value (exclude the regions in which differential values change regularly; column 18, line 
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65 to column 19, line 17); an intensity map configured to indicate true (white region) where an 
intensity value has a high value (corresponding to a low pixel density value; column 17, line 33- 
42). Cosatto (see above discussion of claims 1, 23, and 45) discloses a combined map 
configured to indicate true where information obtained from several channels are combined to 
find features including shape, color, and motion (column 6, Une 20-34), and at least one 
candidate patch is selected from said combined map, wherein said candidate patches have 
suitable hue saturation (skin color; column 13, line 14-29). 

1 1 . Regarding claim 34, Kinjo discloses an apparatus comprising: a processor (facial-region 
extracting unit 40) coupled to a memory (ROM) through a bus; and a detection process 
(program) executed by the processor from the memory (column 16, line 37-48) to cause the 
processor to determine a first area wherein a color gradient (changing differential values) has a 
low value (exclude the regions in which differential values change regularly; column 18, line 65 
to column 19, line 17); determine a second area (white region) wherein an intensity value has a 
high value (corresponding to a low pixel density value; column 17, line 33-42). Cosatto (see 
above discussion of claims 1, 23, and 45) discloses that information obtained from several 
channels are combined ("logical AND") to find feature areas including shape, color, and motion 
(column 6, line 20-34), and selecting portions of the combined feature areas with suitable hue 
saturation (skin color) to form said at least one candidate patch (column 13, hne 14-29). 

12. Regarding claims 2, 13, 24, and 35, Kinjo discloses that determining said first area uses a 
first threshold value comparison (column 39, line 36-44). 

13. Regarding claims 4, 15, 26, and 37, Kinjo discloses that determining said second area 
(density) uses a second threshold value comparison (column 17, line 33-42). 
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14. Regarding claims 3, 5, 14, 16, 25, 27, 36, and 38, Kinjo does not explicitly disclose that 
said fist or said second threshold is determined by normalization. The examiner takes Official 
Notice that determining thresholds by normalization is well known in the art. It would have been 
obvious to one of ordinary skill in the art at the time the invention was made to determine the 
predetermined threshold for binarization and differential density values normalization using the 
average density calculated for the overall image as taught by Kinjo (column 17, line 4-9). 

15. Regarding claims 8, 19, 30, and 41, Kinjo discloses fitting an ellipse (elliptical region) to 
one of said at least one candidate patch (column 21 , line 21-28). 

16. Claims 6, 7, 9-1 1, 17, 18, 20-22, 28, 29, 31-33, 39, 40, and 42-44 are rejected under 35 
U.S.C. 103(a) as being unpatentable over Kinjo in view of Cosatto as applied to claims 1, 8, 12, 
19, 23, 30, 34, and 41 above, and further in view of Lobo et al. (USPN 5,781,650, cited on 
appUcant's IDS), hereafter Lobo. 

17. Regarding claims 6 and 17, neither Kinjo nor Wang disclose that said third area is eroded 
or that said combined map includes an eroded boundary. Lobo discloses a system for automatic 
feature detection and age classification of human faces in digital images including a face 
template that is used for oval- fitting by the use of a potential image of an edge where a 
morphological operator first broadens the image's similar intensity regions, then narrows 
(erodes) the similar intensity regions in a copy of the image, and finally the narrowed image is 
subtracted fi*om the broadened image (column 4, line 44-57). It would have been obvious to one 
of ordinary skill in the art at the time the invention was made to erode similar intensity regions as 
taught by Lobo in order to remove stray pixels and to isolate edges for detection. 
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18. Regarding claims 7 and 18, Lobo discloses that said eroding (narrowing) is 
morphological (column 4, line 49-63). 

19. Regarding claims 9 and 20, neither Kinjo nor Wang expUcitly disclose determining if 
said ellipse is a bad fit to said at least one candidate patch or that said ellipse includes a degree of 
fit measure. Lobo discloses an oval fitting operation including iteratively updating the oval 
center position and the oval axes half-lengths until the total energy stabilizes around a minimum 
value and a final fit has been reached (column 5, line 39-48). It would have been obvious to one 
of ordinary skill in the art at the time the invention was made to determine the degree of fit of an 
ellipse about a candidate patch as taught by Lobo in order to ensure that the ellipse/oval is 
properly positioned and scaled to include the optimum amount of information for personnel 
identification or the like. 

20. Regarding claims 10 and 21, Lobo discloses further processing said at least one candidate 
patch when said ellipse is a bad fit (iteratively update until total energy stabilizes; column 5, line 
39-48). 

21. Regarding claims 1 1 and 22, Kinjo discloses determining if said one of said at least one 
candidate patch is too smooth (column 18, line 54-63). 



22. The prior art made of record and not relied upon is considered pertinent to applicant's 
disclosure. 

• USPN 5,583,659 to Lee et al. discloses a multi-windowing technique for thresholding 



Conclusion 



an image using local image properties including gradient and intensity. 
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• USPN 5,604,823 to Ono et al. discloses a method for extracting object images and 
method for detecting movements thereof including generating a potential field of 
gradients where the center point of a view window is stable at a flat part of the field. 

• USPN 5,751,286 to Barber et al. discloses an image query system and method 
wherein images are searched using visual characteristics such as color, texture, shape, 
size, etc. which may be arithmetically or logically combined. 

• USPN 5,774,177 to Lane discloses a textile fabric inspection system using a mask to 
generate a gradient image, which is then thresholded, dilated, eroded, and subjected to 
a blob analysis. 

• USPN 5,867,609 to Shamoon et al. discloses a method for computing correlation 
operations on partially occluded data using at least two discriminants such as intensity 
and texture in conjunction to locate regions in the image. 

• USPN 6,075,557 to HoUiman et al. discloses an image tracking system and method 
wherein both hue and saturation change smoothly over a large portion of a human 
face under uniform illumination. 

• USPN 6,263,1 13 to Abdel-Mottaleb et al. discloses a method for detecting a face in a 
digital image wherein skin color components having similar features are grouped 
together and further examined. 

• USPN 6,445,835 to Qian discloses a method for image characterization using color 
and texture statistics with embedded spatial information. 
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• USPN 6,272,249 to Kobayashi et al. discloses an image processing device capable of 
discriminating the classification of a particular area using a complexity calculating 
circuit and a difference comparator. 

• Birchfield discloses "Elliptical head tracking using intensity gradients and color 
histograms." 

• Singh et al. discloses "Monitoring driver fatigue using facial analysis techniques" 
including skin color information and gradient maps. 

• Zhang et al. discloses "Locating facial features with color information" including a 
gradient decomposed Hough transform. 

• Birchfield discloses "An elliptical head tracker" including determining the position 
and size of the best ellipse by maximizing the normalized sum of the image gradient 
magnitude around the perimeter of the ellipse. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Ryan J Hesseltine whose telephone number is 703-306-4069. 
The examiner can normally be reached on Monday - Friday, 8:30 AM - 5 PM. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Amelia Au can be reached on 703-308-6604. The fax phone number for the 
organization where this application or proceeding is assigned is (703) 872-9306. 

Any inquiry of a general nature or relating to the status of this application or proceeding 
should be directed to the receptionist whose telephone number is 703-^6-0377. >0 
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October 10, 2003 




